The present invention relates generally to the field of image capture and, more particularly, to panoramic video cameras, camera systems, and methods that provide object tracking-based image capture and image zoom.
Panoramic video cameras are known to capture a field of view (FOV) of 360° about an optical axis of the lens or lenses used in the cameras. Such FOV is typically referred to as the “horizontal FOV” of the camera. Panoramic video cameras can also simultaneously capture an FOV about an axis orthogonal to the optical axis of the camera lens(es). This additional FOV is typically referred to as the “vertical FOV” of the camera. The vertical FOV may exceed 180° when the camera includes one or more ultra-wide angle lenses. The combination of the horizontal FOV and the vertical FOV provides the overall FOV of the panoramic video camera (e.g., 360°×180°, 360°×270°, and so forth). A wide overall FOV permits the camera to capture environmental information about a physical region surrounding the camera. Accordingly, a single panoramic video camera placed in the center of a meeting table is capable of capturing imagery for all participants sitting around the table.
One common use case for a panoramic video camera is to mount the camera on a moving object, such as a person, bicycle, or automobile, to capture imagery of an activity, such as skiing, surfing, bike riding, auto racing, and the like. A goal of such use is to permit playback of the captured video on a display that enables a viewer to become immersed in the experience. A user often is provided controls to alter a view of video playback, where the playback includes a relative center. That is, playback typically provides a horizontal FOV viewing segment of approximately 110° of an available 360° arc. For playback, a center for this viewing segment must be defined, which by default is typically a vector consistent with a relative motion of the camera itself.
Often, there is a desire for real-time or near real time playback (e.g., live streaming) of video from a panoramic video camera. This rapid response complicates intra-camera processing tremendously because it requires camera processor operations to be split between image capture and playback. To ensure effective end-user experiences at playback, some minimal overhead is necessary, which is often handled internally by camera hardware. For example, the horizontal FOV viewing segment for playback video is a sub-portion (e.g., 110° degree arc) of the horizontal FOV for the captured image content (e.g., 360° arc). Additionally, the horizontal FOV viewing segment boundaries may require adjusting to stabilize playback. That is, because the camera itself is in motion due to being mounted to a dynamically moving object, a smoothing function may need to be applied to compensate for camera based motion. Overall, processor intensive digital signal processing (DSP) operations must be minimized to minimize latency between video capture and playback.
Typical zoom functionality implemented for panoramic video cameras and playback systems relegate any zooming to a post-process (pre-playback) function controlled by a user playback interface. A user is presented with a zoom option, which alters playback magnification per user input. Software established digital zoom is a limited process practically constrained by a fidelity of image data contained in an underlying video segment. A balance of competing factors exists for optimal user experience regarding playback based zoom functionality. Video size is generally minimized to facilitate rapid transfer of video content and playback of the same, as the greater the size of a video segment file, the more resource consumption for transport and processing. To enable accurate zooming that does not rapidly pixelate upon magnification, a video segment file pixel density is increased to ensure. Sometimes, smoothing functions are implemented to add additional pixels not present in the file to improve appearance during zooming operations at an expense of accuracy and additional processing resources. Regardless of technology being implemented, a balance between competing interests occurs. Additionally, implementation of a manual zoom feature imposes a manual overhead on an end-user.
Those skilled in the field of the present disclosure will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. The details of well-known elements, structure, or processes that would be necessary to practice the embodiments, and that would be well known to those of skill in the art, are not necessarily shown and should be assumed to be present unless otherwise indicated.